Position detection and/or movement tracking via image capture and processing

ABSTRACT

Position detection and/or movement tracking via image capture and processing. Digital cameras perform image capture of one or more objects within a particular region (e.g., a physical gaming environment). A game module or processing module processes the images captured by the digital cameras to identify a position of and/or track movement of objects (e.g., a player, a gaming object, a game controller, etc.). Various digital image processing techniques may be employed including pattern recognition of objects, color recognition/distinction, intensity recognition/distinction, relative size comparison, etc. to identify objects and/or track their movement. The coupling between the digital cameras and the game module or processing module may be wired, wireless, or a combination thereof. If wireless, any number of different signaling means may be employed including Code Division Multiple Access (CDMA) signaling, Time Division Multiple Access (TDMA) signaling, or Frequency Division Multiple Access (FDMA) signaling.

CROSS REFERENCE TO RELATED PATENTS/PATENT APPLICATIONS ProvisionalPriority Claims

The present U.S. Utility Patent Application claims priority pursuant to35 U.S.C. §119(e) to the following U.S. Provisional Patent Applicationwhich is hereby incorporated herein by reference in its entirety andmade part of the present U.S. Utility Patent Application for allpurposes:

1. U.S. Provisional Application Ser. No. 60/936,724, entitled “Positionand motion tracking of an object,” (Attorney Docket No. BP6471), filedJun. 22, 2007, pending.

BACKGROUND OF THE INVENTION

1. Technical Field of the Invention

The invention relates generally to position and tracking systems; and,more particularly, it relates to such systems that employ captureddigital images to determine position of or track movement of an object.

2. Description of Related Art

Communication systems are known to support wireless and wire linedcommunications between wireless and/or wire lined communication devices.Such communication systems range from national and/or internationalcellular telephone systems to the Internet to point-to-point in-homewireless networks to radio frequency identification (RFID) systems. Eachtype of communication system is constructed, and hence operates, inaccordance with one or more communication standards. For instance, radiofrequency (RF) wireless communication systems may operate in accordancewith one or more standards including, but not limited to, RFID, IEEE802.11, Bluetooth, advanced mobile phone services (AMPS), digital AMPS,global system for mobile communications (GSM), code division multipleaccess (CDMA), local multi-point distribution systems (LMDS),multi-channel-multi-point distribution systems (MMDS), and/or variationsthereof. As another example, infrared (IR) communication systems mayoperate in accordance with one or more standards including, but notlimited to, IrDA (Infrared Data Association).

Depending on the type of RF wireless communication system, a wirelesscommunication device, such as a cellular telephone, two-way radio,personal digital assistant (PDA), personal computer (PC), laptopcomputer, home entertainment equipment, RFID reader, RFID tag, et ceteracommunicates directly or indirectly with other wireless communicationdevices. For direct communications (also known as point-to-pointcommunications), the participating wireless communication devices tunetheir receivers and transmitters to the same channel or channels (e.g.,one of the plurality of radio frequency (RF) carriers of the wirelesscommunication system) and communicate over that channel(s). For indirectwireless communications, each wireless communication device communicatesdirectly with an associated base station (e.g., for cellular services)and/or an associated access point (e.g., for an in-home or in-buildingwireless network) via an assigned channel. To complete a communicationconnection between the wireless communication devices, the associatedbase stations and/or associated access points communicate with eachother directly, via a system controller, via the public switch telephonenetwork, via the Internet, and/or via some other wide area network.

For each RF wireless communication device to participate in wirelesscommunications, it includes a built-in radio transceiver (i.e., receiverand transmitter) or is coupled to an associated radio transceiver (e.g.,a station for in-home and/or in-building wireless communicationnetworks, RF modem, etc.). As is known, the receiver is coupled to theantenna and includes a low noise amplifier, one or more intermediatefrequency stages, a filtering stage, and a data recovery stage. The lownoise amplifier receives inbound RF signals via the antenna andamplifies then. The one or more intermediate frequency stages mix theamplified RF signals with one or more local oscillations to convert theamplified RF signal into baseband signals or intermediate frequency (IF)signals. The filtering stage filters the baseband signals or the IFsignals to attenuate unwanted out of band signals to produce filteredsignals. The data recovery stage recovers raw data from the filteredsignals in accordance with the particular wireless communicationstandard.

As is also known, the transmitter includes a data modulation stage, oneor more intermediate frequency stages, and a power amplifier. The datamodulation stage converts raw data into baseband signals in accordancewith a particular wireless communication standard. The one or moreintermediate frequency stages mix the baseband signals with one or morelocal oscillations to produce RF signals. The power amplifier amplifiesthe RF signals prior to transmission via an antenna.

In most applications, radio transceivers are implemented in one or moreintegrated circuits (ICs), which are inter-coupled via traces on aprinted circuit board (PCB). The radio transceivers operate withinlicensed or unlicensed frequency spectrums. For example, wireless localarea network (WLAN) transceivers communicate data within the unlicensedIndustrial, Scientific, and Medical (ISM) frequency spectrum of 900 MHz,2.4 GHz, and 5 GHz. While the ISM frequency spectrum is unlicensed thereare restrictions on power, modulation techniques, and antenna gain.

In IR communication systems, an IR device includes a transmitter, alight emitting diode, a receiver, and a silicon photo diode. Inoperation, the transmitter modulates a signal, which drives the LED toemit infrared radiation which is focused by a lens into a narrow beam.The receiver, via the silicon photo diode, receives the narrow beaminfrared radiation and converts it into an electric signal.

IR communications are used video games to detect the direction in whicha game controller is pointed. As an example, an IR sensor is placed nearthe game display, where the IR sensor to detect the IR signaltransmitted by the game controller. If the game controller is too faraway, too close, or angled away from the IR sensor, the IR communicationwill fail.

Further advances in video gaming include three accelerometers in thegame controller to detect motion by way of acceleration. The motion datais transmitted to the game console via a Bluetooth wireless link. TheBluetooth wireless link may also transmit the IR direction data to thegame console and/or convey other data between the game controller andthe game console.

While the above technologies allow video gaming to include motionsensing, it does so with limitations. As mentioned, the IR communicationhas a limited area in which a player can be for the IR communication towork properly. Further, the accelerometer only measures accelerationsuch that true one-to-one detection of motion is not achieved. Thus, thegaming motion is limited to a handful of directions (e.g., horizontal,vertical, and a few diagonal directions).

Therefore, a need exists for motion tracking and positioningdetermination for video gaming and other applications that overcome theabove limitations.

BRIEF SUMMARY OF THE INVENTION

The present invention is directed to apparatus and methods of operationthat are further described in the following Brief Description of theSeveral Views of the Drawings, the Detailed Description of theInvention, and the claims. Other features and advantages of the presentinvention will become apparent from the following detailed descriptionof the invention made with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a diagram of an embodiment an apparatus that performs positiondetermination and/or movement tracking via image capture and processing.

FIG. 2 is a diagram of an alternative embodiment of an apparatus thatperforms position determination and/or movement tracking via imagecapture and processing.

FIG. 3 is a diagram of an embodiment showing a means by which positionof a point, object, etc. may be determined using multiple directionalvectors extending from multiple known locations, respectively, to thatpoint, object, etc.

FIG. 4 is a diagram of an embodiment showing the relationship between anobject point and various image planes that have performed image captureof the object point.

FIG. 5 is a diagram of an embodiment showing the relationship betweenmultiple object points and various image planes that have performedimage capture of the multiple object points.

FIG. 6 is a diagram of an embodiment showing an image sensor and theassociation of physical pixels and the image pixels generated therefrom.

FIG. 7A and FIG. 7B are diagrams of an embodiment of an apparatus thatemploys directional vectors associated with captured images, at leastsome of which depict an object, to determine position of the object.

FIG. 8A and FIG. 8B are diagrams of an embodiment of an apparatus thatemploys directional vectors associated images, that depict a number ofobjects, to determine position of a device that has captured the images.

FIG. 9 is a schematic block diagram of an overhead view of an embodimentof a gaming system.

FIG. 10 is a schematic block diagram of a side view of an embodiment ofa gaming system.

FIG. 11 is a diagram illustrating an embodiment of a gaming systemincluding multiple digital cameras for capturing images to undergoprocessing in a game module, that is wire-coupled to the multipledigital cameras, for position detection and/or movement tracking.

FIG. 12 is a diagram illustrating an alternative embodiment of a gamingsystem including multiple digital cameras for capturing images toundergo processing in a game module, that is wirelessly coupled to atleast some of the multiple digital cameras, for position detectionand/or movement tracking.

FIG. 13 is a schematic block diagram of a side view of anotherembodiment of a gaming system.

FIG. 14 is a schematic block diagram of an overhead view of anotherembodiment of a gaming system.

FIG. 15, FIG. 16, and FIG. 17 are diagrams of an embodiment of acoordinate system of a gaming system.

FIG. 18, FIG. 19, and FIG. 20 are diagrams of another embodiment of acoordinate system of a gaming system.

FIG. 21 is a diagram of a method for determining position and/or motiontracking.

FIG. 22 is a diagram of another method for determining position and/ormotion tracking.

FIG. 23, FIG. 24, and FIG. 25 are diagrams of another embodiment of acoordinate system of a gaming system.

FIGS. 26, FIG. 27, and FIG. 28 are diagrams of another embodiment of acoordinate system of a gaming system.

FIG. 29 is a diagram of another method for determining position and/ormotion tracking.

FIG. 30 is a diagram of another method for determining position and/ormotion tracking.

FIG. 31 is a diagram of another method for determining position and/ormotion tracking.

FIG. 32 is a diagram of another method for determining position and/ormotion tracking.

FIG. 33 is a diagram of another embodiment of a coordinate system of agaming system.

FIG. 34 is a diagram of a method for determining motion.

FIG. 35 is a diagram of an example of reference points on a playerand/or gaming object.

FIG. 36, FIG. 37, and FIG. 38 are diagrams of examples of motionpatterns.

FIG. 39 is a diagram of an example of motion estimation.

FIG. 40 and FIG. 41 are diagrams of examples of reference points on aplayer to determine player's physical measurements.

FIG. 42 is a diagram of an example of mapping a player to an image.

FIG. 43 is a diagram of another method for determining motion.

FIG. 44 is a schematic block diagram of an embodiment of a gaming objectand/or game console.

FIG. 45, FIG. 46, and FIG. 47 are diagrams of various embodiments ofmethods for determining position and/or motion tracking.

FIG. 48 is a diagram of an embodiment of a method for determining adistance based on captured digital images.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a diagram of an embodiment an apparatus that performs positiondetermination and/or movement tracking via image capture and processing.The apparatus includes a number of digital cameras that generate digitalimages. An object is depicted within at least some of the digitalimages. A processing module is coupled to receive the digital images.The processing module processes the digital images to identifycharacteristics of the object as depicted within at least some of thedigital images. Based on the identified characteristics, the processingmodule determines position of the object with respect to locations of atleast some of the digital cameras.

In one embodiment, the processing module identifies directional vectorsbased on the identified characteristics of the object. These directionalvectors may be viewed as extending from known locations (e.g., locationsof the digital cameras, point of reference within the digital cameras,etc.) to the object. In the context of using a digital camera, a digitalcamera includes an electronic image sensor. A digital image sensor, whenmounted on a surface of an integrated circuit and implemented forperforming image capture directly may also be viewed as an alternativeembodiment of a digital camera.

The specifications of such digital image sensors are oftentimes definedin terms of number of physical pixels within the digital image sensorthat correspond to the number of image pixels that a picture captured bythe image sensor will have. For example, as processes by which digitalcameras are manufactured continues to improve, the number of mega-pixelsthat a digital image sensor includes continues to increase. Generally,the digital image sensors within digital cameras have more than amillion physical pixels (e.g., mega-pixels (or more)).

A reference point within a digital camera may serve as a point fromwhich a directional vector is defined. As one example, when an image iscaptured by a digital camera, a camera center of projection of thedigital camera is a point to which all points in the image can be tracedback to. The focal distance of the digital camera may also correspond tothe camera center of projection of the digital camera. A directionalvector may be defined as extending from such a reference point withinthe digital camera to a physical pixel that has captured a particularportion of an object of interest. In other words, an image pixel ofinterest within a digital image corresponds to a physical pixel of thedigital image sensor of the digital camera. A directional vector may bedefined as extending from that reference point within the digital camerato that physical pixel.

Any of a variety of means may be performed to identify thecharacteristics of the object depicted within at least some of thedigital images, including any of a variety of pattern recognitionprocesses. Moreover, an object may include one or more sensing tagsthereon to assist in the identification of the characteristics of theobject depicted within at least some of the digital images.

Some examples of sensing tags include a particular type of material(e.g., metal, etc.), an RFID tag, a material having particularproperties (e.g., a light reflective material, a light absorbentmaterial, etc.), a specific RGB [red, green, blue] color or combinationof colors, a particular pattern, etc.). By discerning and distinguishingdifferent sensing tags that may be placed on different parts of theobject, the relative position of those parts of the object may bedetermined. This may be performed in addition to the overall position ofthe object that may be determined by identifying the entire object.

In addition, the object whose characteristics are identified may have apredetermined size. In some of the embodiments depicted herein, aplayer/user may employ a gaming object when playing a game, and the sizeof such a gaming object may be known beforehand. When an object having apredetermined size is identified in a digital image, then theactual/physical size of the object may be associated with the identified‘image size’ as depicted within the digital image. The relationshipbetween these two (e.g., image size and predetermined size) may beemployed to determine a scaling factor for that digital image. With thisinformation, a distance between two objects depicted within the digitalimage may be determined.

Moreover, it is noted that once the position of the object is known,then that position may be mapped to a virtual 3D (three-dimensional)coordinate system. This may be employed within a variety of systemsincluding a gaming system such as is described herein.

Each of the digital cameras has a corresponding field of view in whichit can perform image capture. Again, the object is depicted within atleast some of the fields of view of at least some of the digitalcameras. When the object is not within any field of view of any digitalcamera, then at least some of the digital cameras can be adjusted (e.g.,such as using an actuator coupled to or integrated with a digitalcamera) so that the object may be visible within at least one of thefields of view of at least one of the cameras.

It is also noted that the configuration of any of the digital camerasmay be adjusted. For example, a digital camera may have auto-focuscapability in which the focal distance of the digital camera is adjustedto provide a maximum clarity image of the object of interest. Moreover,the image capture rate of any digital camera may be adjusted based on anumber of factors including a predetermined setting within theprocessing module, a user-selected setting within the processing module,a movement history of the object, a current movement of the object, andan expected future movement of the object.

It is noted that, while position determination is described herein withrespect to an object, the movement of the object may also be determinedby merely updating the position of the object as a function of time. Forexample, the processing module may determine a first position of theobject during a first time, and the processing module may then determinea second position of the object during a second time. The movement ofthe object may be estimated by comparing the first determined positionand the second determined position. The rate of the movement of theobject may be determined by also considering the times associated withthe each of the first determined position and the second determinedposition.

It is also noted that the digital cameras may be ‘smart’ digital camerasin some embodiments that include means by which the configuration of thedigital camera may be determined and communicated back to the processingmodule. Certain information such as focal length of the digital camera,the image capture setting of the digital camera (e.g., for digitalcameras that can capture images having different numbers of pixels),physical orientation, physical location, etc. may be determined by sucha smart digital camera, communicated back to the processing module, andthen the processing module can consider this higher level of informationwhen employing the identified characteristics of the object to determinethe position of the object.

Moreover, it is noted that while wire-coupling between the directionalmicrophones and the processing module are illustrated in thisembodiment, wireless communication may also employed between the variouscomponents of such an apparatus without departing from the scope andspirit of the invention.

FIG. 2 is a diagram of an alternative embodiment of an apparatus thatperforms position determination and/or movement tracking via imagecapture and processing. This embodiment is somewhat analogous to theprevious embodiment, with at least one difference being that the digitalcameras are wirelessly coupled to the processing module. It is alsonoted that at least one digital camera may be integrated into theprocessing module.

The wireless means by which communication is supported may be varied,and it may be supported using any desired radio frequency (RF)communication standard including any that operates in accordance withone or more standards including, but not limited to, RFID, IEEE 802.11,Bluetooth, advanced mobile phone services (AMPS), digital AMPS, globalsystem for mobile communications (GSM), code division multiple access(CDMA), local multi-point distribution systems (LMDS),multi-channel-multi-point distribution systems (MMDS), and/or variationsthereof.

Moreover when the use of RF communication is employed within such anapparatus, at least one of the digital cameras includes a first radiofrequency (RF) transceiver, and the processing module includes a secondRF transceiver. Based on an RF signal transmitted between the first RFtransceiver and the second RF transceiver, the processing module canthen determine a distance between the processing module and the digitalcamera from which the RF signal was transmitted. By using a transmissiontime at which the RF signal is transmitted from a first device, and areceive time at which the RF signal is received by a second device, andalso knowing the speed/velocity at which the RF signal travels, then thedistance between the first device and the second device may bedetermined.

FIG. 3 is a diagram of an embodiment showing a means by which positionof a point, object, etc. may be determined using multiple directionalvectors extending from multiple known locations, respectively, to thatpoint, object, etc. This diagram depicts 3D space in a right handed,Cartesian coordinate system (e.g., shown as having axes xyz). Clearly,the principles described with respect to this diagram are applicable toany other 3D coordinate system as well.

When at least two positions are known, and when directional vectorsextending from each of those two locations are known, then if thosedirectional vectors do intersect at all, then the location of theintersection may be determined using triangulation. If additional knownlocations are known, and if additional directional vectors extendingfrom those additional known locations are also known, then a greatercertainty of an intersection between the various directional vectors maybe had.

It is noted that once the position associated with the intersection ofthese directional vectors is known, then this position (or location) maybe mapped to a virtual 3D coordinate system. The upper right hand cornerof the diagram depicts a virtual 3D space in a right handed, Cartesiancoordinate system (e.g., shown as having axes x′y′z′).

FIG. 4 is a diagram of an embodiment showing the relationship between anobject point and various image planes that have performed image captureof the object point. This diagram shows two separate image planes, ascorresponding to two separate digital cameras, that capture digitalimages of an object from different perspectives or fields of view. Theimage plane of a digital camera may be considered as corresponding tothe digital image sensor component of the digital camera. For example, adigital mage sensor may be a complementary metal-oxide-semiconductor(CMOS) device or a charge coupled device (CCD). As is known, variousparameters generally are employed to define a digital image sensor,including an image sensor type (e.g., ¼″, 1/3.6″, etc.), a width andheight (typically provided in milli-meters), a total number of physicalpixels (e.g., X megapixels, where X is a number such as 3, 6, 8.1,etc.), a number of physical pixels along each of the width and height ofthe digital image sensor (e.g., y×z, where y and z are integer numbers),a diagonal size (again, typically provided in milli-meters) thatcorresponds to the normal lens focal length, the focal length factor,etc. the general trend in digital image sensor development over theyears is to pack more and more physical pixels into a digital imagesensor while also trying to reduce the overall size of the digital imagesensor. In any case, each physical pixel of a digital image sensorcaptures information (e.g., color, intensity, etc.) of a portion of thefield of view of the digital camera, and this information is employed togenerate an image pixel of a digital image. Therefore, in the digitalimage context, there can be viewed as being a one to one relationshipbetween each physical pixel of a digital image sensor and each imagepixel of an image generated from information captured by the digitalimage sensor.

In this diagram, a directional vector extends from a reference point ofa digital camera 1 (DC1) through the image plane of DC1 to a point onthe object of interest. As can be seen, a directional vector (DV1) alsoextends from this DC1 reference point through the image plane of DC1(e.g., which corresponds to the digital image sensor of DC1. This camerareference point may be a camera center of projection for DC1 based onits current configuration (e.g., focus, etc.). Alternatively, anothercamera reference point may be employed (e.g., focal point, predeterminedpoint within the camera, etc.) without departing from the scope andspirit of the invention.

Analogously for a second digital camera (DC2), another directionalvector extends from a reference point of a DC2 through the image planeof DC2 to the same point on the object of interest. If the locations ofDC1 and DC2 are known, and if the directional vectors extending from therespective points of reference of each of DC1 and DC2 are known, thenthe principles of triangulation may be employed to determine thelocation of the object point on the object of interest.

As can also be seen in this diagram, there is a relationship between thedimensions of object (physically) and the corresponding images of thatobject as depicted in the digital images captured by DC1 and DC2. Forone example, when considering the actual height of the object, then animage 1 height is the height of the object as depicted in a digitalimage captured by DC1, and an image 2 height is the height of the objectas depicted in a digital image captured by DC2. These two image heightsneed not be the same (e.g., the object may be closer to one of thedigital cameras than the other, the focus of one of the digital camerasmay be different than the other, etc.). It is noted that if the actualheight of the object is known, then a first ratio between the actualheight to the image 1 height may be made, and a second ratio between theactual height to the image 2 height may be made. By knowing the actualsize of something depicted within a digital image, and by knowing theconfiguration of the digital camera (e.g., focus, etc.), then a distancebetween the digital camera and the object may be determined.

FIG. 5 is a diagram of an embodiment showing the relationship betweenmultiple object points and various image planes that have performedimage capture of the multiple object points. This diagram has somesimilarities to the previous embodiment, in that a directional vectorextends from a reference point of a digital camera through the imageplane of the digital camera to a point on the object of interest.

However, the object of this embodiment includes a number of sensing tagsthereon. These sensing tags can be portions of the object having aparticular color, a light reflective material, a light absorbentmaterial, an infrared light source, etc. Generally, the sensing tagshave some associated characteristic that is identifiable on the object.

The object in this diagram also has different types of sensing tags(e.g., of type 1, type 2, etc.). This use of different types of sensingtags of an object may be employed to assist in determining the positionand orientation of the object (e.g., sometimes referred to as ‘pose’ inthe image processing context), since different sides, areas, etc. of theobject may be better distinguished from one another. For example, whenconsidering an object such as a cube, then a determination of whetherthe cube is right side up (or upside down) with reference to a desiredconvention of which side of the cube will be deemed to be ‘up’ may bedetermined.

In this embodiment, first directional vectors associated with type 1sensing tags extend from a reference point of a digital camera throughthe image plane of the digital camera to two separate points on theobject that have type 1 sensing tags. Second directional vectorsassociated with type 2 sensing tags extend from the reference point ofthe digital camera through the image plane of the digital camera to twoseparate points on the object that have type 2 sensing tags.

FIG. 6 is a diagram of an embodiment showing an image sensor and theassociation of physical pixels and the image pixels generated therefrom. Within a digital camera, a digital image sensor is the elementthat captures information (e.g., color, intensity, contrast, etc.) of afield of view of the digital camera. Each individual physical pixel ofthe digital image sensor captures a small portion of the field of viewof the digital camera. For example, if the digital image sensor includesone million physical pixels, then each individual physical pixel of thedigital image sensor captures information of one-millionth of the fieldof view of the digital camera. If the digital image sensor includes Xmegapixels, then each individual physical pixel of the digital imagesensor captures information of (1/(X×10⁶))^(th) of the field of view ofthe digital camera.

Together, each of these discrete pieces of information, as captured bythe physical pixels, is used to form a digital image corresponding whatis seen in the field of view of the digital camera.

A directional vector extends from a reference point of a digital camerato one of the physical pixels of the digital image sensor. For example,when a particular image pixel of a digital image is identified, then thecorresponding physical pixel that captured information used to generatethat image pixel can be determined. Such a directional vector can thenbe determined. This directional vector may be the directional vectorgenerated from this digital camera to a particular point on the objectof interest.

FIG. 7A and FIG. 7B are diagrams of an embodiment of an apparatus, shownfrom two separate perspectives, that employs directional vectorsassociated with captured images, at least some of which depict anobject, to determine position of the object.

Referring to perspective of FIG. 7A, which is viewed in the xy plane ofa 3D space having an xyz coordinate system, the principles of usingtriangulation may be employed when determining position of an objectthat is depicted in digital images captured by multiple digital cameras.For example, a projection of a first directional vector (DV1 proj.) froma first digital camera (DC1) extends from the first digital camera tothe object. A projection of a second directional vector (DV2 proj.) froma second digital camera (DC2) extends from the second digital camera tothe object. Additional directional vectors, associated with additionaldigital cameras, may also be employed. The directional vectors thenundergo processing in a processing module to determine the intersectionof the various directional vectors. The intersection of thesedirectional vectors is the location of the object.

Referring to perspective of FIG. 7B, this diagram is viewed in the xzplane of a 3D space having an xyz coordinate system.

FIG. 8A and FIG. 8B are diagrams of an embodiment of an apparatus, shownfrom two separate perspectives, respectively, that employs directionalvectors associated images, that depict a number of objects, to determineposition of a device that has captured the images.

Referring to the embodiment of FIG. 8A, which is viewed in the xy planeof a 3D space having an xyz coordinate system, the principles of usingtriangulation may be employed when determining position of a device thatincludes multiple digital cameras (e.g., a first digital camera (DC1), asecond digital camera (DC2), etc.) that capture digital images thatdepict various known objects (e.g., a first object (object 1), a secondobject (object 2), etc.).

The principles of triangulation are employed in this embodiment, but inreverse that the previous embodiment. The orientation of each digitalcamera of the device, when capturing a digital image of a known objectis determined.

For example, a projection of a first directional vector (DV1 proj.) froma first object (object 1) extends to the first digital camera (DC1). Aprojection of a second directional vector (DV2 proj.) extends from asecond object (object 2) to a second digital camera (DC2). Additionaldirectional vectors, associated with additional objects, may also beemployed. The directional vectors orientations undergo processing in aprocessing module to determine their intersection. The intersection ofthese directional vectors is the location of the device that includesthe multiple digital cameras.

Referring to the embodiment of FIG. 8B, this diagram is viewed in the xzplane of a 3D space having an xyz coordinate system.

FIG. 9 is a schematic block diagram of an overhead view of an embodimentof a gaming system that includes a game console and a gaming object. Thegaming system has an associated a physical area in which the gameconsole and the gaming object are located. The physical area may be aroom, portion of a room, and/or any other space where the gaming objectand game console are proximally co-located (e.g., airport terminal, on abus, on an airplane, etc.).

The gaming object may be a wireless game controller and/or any objectused or worn by the player to facilitate play of a video game. Forexample, the gaming object may be a simulated sword, a simulated gun, ahelmet, a vest, a hat, shoes, socks, pants, shorts, gloves, etc.

In this system, the game console determines the positioning of thegaming object within the physical area using one or more positiondetermination techniques as subsequently discussed. Once the gamingobject's position is determined, the game console tracks the motion ofthe gaming object using one or more motion tracking techniques assubsequently discussed to facilitate video game play. In thisembodiment, the game console may determine the positioning of the gamingobject within a positioning tolerance (e.g., within a meter) at apositioning update rate (e.g., once every second or once every fewseconds) and tracks the motion within a motion tracking tolerance (e.g.,within a few millimeters) at a motion tracking update rate (e.g., onceevery 10-100 milliseconds).

FIG. 10 is a schematic block diagram of a side view of an embodiment ofa gaming system of FIG. 9 to illustrate that the positioning and motiontracking are done in three-dimensional space. As such, the gaming systemprovides accurate motion tracking of the gaming object, which may beused to map the player's movements to a graphics image for trueinteractive video game play.

FIG. 11 is a diagram illustrating an embodiment of a gaming systemincluding multiple digital cameras for capturing images to undergoprocessing in a game module, that is wire-coupled to the multipledigital cameras, for position detection and/or movement tracking. Aphysical gaming environment (at least a portion of which may berepresented within a virtual gaming environment) includes a number ofdigital cameras arranged at various locations therein to effectuate theimage capture of a player and/or gaming object associated with theplayer. There may be some instances where the player has no gamingobject (e.g., when simulating boxing), and the bodily position and/ormovement of the player are those elements being monitored and/ortracked.

Each digital camera has a corresponding field of view in which it canperform image capture. By appropriately placing the digital camerasthroughout various locations in an area, an entirety of the physicalgaming environment can be visually captured by digital images generatedby the digital cameras. By crossing more than one field of view of morethan one digital camera, then multiple views of a single object withinthe physical gaming environment can be obtained. The game module (oranother processing module) may then process the digital images capturedby the digital cameras to make estimates of a position of an objectwithin the physical gaming environment. Also, by comparing variousdigital images taken at different times (e.g., digital image 1 taken attime 1, digital image 2 taken at time 2=time (1+Δt)), then movement ofthe object within the physical gaming environment may be estimated.

The game console is operable to perform processing of digital imagescaptured by the digital cameras to identify characteristics of an objectdepicted within at least some of the digital images. Based on theidentified object characteristics, the game console is operable todetermine position of the object with respect to the digital cameras.

Moreover, it is noted that, in this embodiment as well as otherembodiments, certain initialization processes can be performed in whichthe player and/or gaming object remains motionless. The digital camerasthen may perform image capture of the motionless player and/or gamingobject for calibration purposes. In addition, if a size (e.g., height,width, etc.) of the player and/or gaming object is known and provided tothe game console (e.g., by being entered via a user interface by theplayer, or by being estimated by the game console), then the size ofother objects within the physical gaming environment may be estimatedbased on their relatively proportional size to a known object.

Also, various means of performing digital image processing may beperformed including pattern recognition in which a predetermined pattern(e.g., as corresponding to a particular shape) is compared to patternsdetected within one of the digital images captured by one of the digitalcameras. It is noted that a particular shape may have more than onepattern corresponding thereto (e.g., a pattern 1 of aperson-related-shape corresponding to a taller/slender person vs. apattern 2 of a person-related-shape corresponding to a shorter/bulkyperson, etc.). Also, it is noted that a pattern detected within adigital image, even if is not an expected pattern or can be associatedwith a predetermined pattern that is being searched for within thedigital image, the detected pattern can be added (e.g., to a memory)that stores a number of patterns/shapes that may be detected within thedigital image.

Another means of performing digital image processing may includesearching for a particular color (e.g., as associated with a player,gaming object, etc.) within a digital image captured by a digitalcamera. For example, a player may wear a particular colored clothingarticle, and when processing the digital image captured by a digitalcamera, the color associated with that known-colored clothing article issought for.

Other means of performing digital image processing may be performedincluding searching for reflections off of reflective material thatcovers the player and/or gaming object. This digital image processingmay involve searching for pixels or groups of pixels within a digitalimage above a certain threshold (which may be predetermined oradaptively set for each digital image). When the intensity is above thatthreshold, then that pixel (or group of pixels) can be associated asbeing associated with the reflective material covering the player and/orgaming object. Additional variations of the physical gaming environmentmay be employed such as providing special lighting to enhance thereflecting of light off of reflective material covering at least aportion of the player and/or gaming object. Moreover, an appropriatebackdrop could also be employed to provide a higher degree of contrastbetween the player and/or gaming object and the rest of the physicalgaming environment.

Certain operational parameters of the digital cameras may also beadjusted by a user/player or in real time by control signals provided bythe game console. For example, the image capture rate employed by thedigital cameras may be adjusted to based on any number of considerationsincluding a predetermined setting within the game console, aplayer-selected setting within the game console (e.g., as selected bythe player via a user interface), a type of game being played, amovement history of the player and/or gaming object, a current orexpected movement of the player and/or gaming object, etc. Also, the anyone of the digital cameras may include an integrated actuator to performreal-time re-positioning of a digital camera to effectuate better imagecapture of the player and/or gaming object within the physical gamingenvironment. Alternatively, the camera may be mounted on an actuatorthat can perform such re-positioning of the digital camera. Clearly, aplayer/user can perform re-positioning of any digital camera as well.

As can be seen in this embodiment, the digital cameras are allwire-coupled to the game console. Any desired wire-based communicationprotocol (e.g., Ethernet) may be employed to effectuate communicationbetween the digital cameras and the game console to communicate digitalimages from the digital cameras to the game console and command signals(if necessary) from the game console to the digital cameras.

FIG. 12 is a diagram illustrating an alternative embodiment of a gamingsystem including multiple digital cameras for capturing images toundergo processing in a game module, that is wirelessly coupled to themultiple digital cameras, for position detection and/or movementtracking.

This embodiment is somewhat analogous to the previous embodiment, withat least one difference being that at least some of the digital camerasand the game console each include wireless communication capability toeffectuate wireless communication there between. In this embodiment, atleast one of the digital cameras is wire-coupled to the game console.For example, some of the digital cameras and the game console eitherincludes an integrated wireless transceiver or is coupled to a wirelesstransceiver to effectuate communication between some of the digitalcameras and the game console. In addition, a digital camera may beintegrated into the game console as well without departing from thescope and spirit of the invention.

This wireless communication can be supported using any number of desiredwireless protocols including Code Division Multiple Access (CDMA)signaling, Time Division Multiple Access (TDMA) signaling, FrequencyDivision Multiple Access (FDMA) signaling, or some other desiredwireless standard, protocol, or proprietary means of communication.

In addition, the wireless communication can be supported using anydesired radio frequency (RF) communication standard including any thatoperates in accordance with one or more standards including, but notlimited to, RFID, IEEE 802.11, Bluetooth, advanced mobile phone services(AMPS), digital AMPS, global system for mobile communications (GSM),code division multiple access (CDMA), local multi-point distributionsystems (LMDS), multi-channel-multi-point distribution systems (MMDS),and/or variations thereof.

FIG. 13 is a schematic block diagram of a side view of anotherembodiment of a gaming system that includes multiple gaming objects, theplayer, and a game console. In this embodiment, the gaming objectsinclude one or more sensing tags (e.g., metal, RFID tag, lightreflective material, light absorbent material, a specific RGB [red,green, blue] color, etc.). For example, the gaming objects may include agame controller, a helmet, a shirt, pants, gloves, and socks, each ofwhich includes one or more sensing tags. In this manner, the sensingtags facilitate the determining of position and/or facilitate motiontracking as will be subsequently discussed.

FIG. 14 is a schematic block diagram of an overhead view of anotherembodiment of a gaming system that includes a game console, a pluralityof players and a plurality of gaming objects. In this instance, thepositioning and motion tracking of each of the gaming objects (and hencethe player) are determined by the game console and/or the one or moreperipheral sensors.

FIG. 15, FIG. 16, and FIG. 17 are diagrams of an embodiment of acoordinate system of a localized physical area that may be used for agaming system. In these diagrams, an xyz origin is selected to besomewhere in the localized physical area and each point being trackedand/or used for positioning on the player and/or on the gaming object isdetermined based on its Cartesian coordinates (e.g., x1, y1, z1). As theplayer and/or gaming object moves, the new position of the trackingand/or positioning points are determined in Cartesian coordinates withrespect to the origin.

FIG. 18, FIG. 19, and FIG. 20 are diagrams of another embodiment of acoordinate system of a localized physical area that may be used for agaming system. In these diagrams, an origin is selected to be somewherein the localized physical area and each point being tracked and/or usedfor positioning on the player and/or on the gaming object is determinedbased on its vector, or spherical, coordinates (ρ, φ, θ), which aredefined as: ρ≧0 is the distance from the origin to a given point P.0≧φ≧180° is the angle between the positive z-axis and the line formedbetween the origin and P. 0≧θ≧360° is the angle between the positivex-axis and the line from the origin to the P projected onto thexy-plane. φ is referred to as the zenith, colatitude or polar angle,while θ is referred to as the azimuth.φ and θ lose significance when ρ=0and θ loses significance when sin(φ)=0 (at φ=0 and φ=180°). To plot apoint from its spherical coordinates, go ρ units from the origin alongthe positive z-axis, rotate φ about the y-axis in the direction of thepositive x-axis and rotate θ about the z-axis in the direction of thepositive y-axis. As the player and/or gaming object moves, the newposition of the tracking and/or positioning points are determined invector, or spherical, coordinates with respect to the origin.

While FIGS. 15-20 illustrate two types of coordinate system, anythree-dimensional coordinate system may be used for tracking motionand/or establishing position within a gaming system.

FIG. 21 is a diagram of a method for determining position and/or motiontracking that begins by determining the environment parameters (e.g.,determining the properties of the localized physical area such asheight, width, depth, objects in the physical area, etc.). The methodthen continues by mapping the environment parameters to a coordinatesystem (e.g., Cartesian coordinate system of FIGS. 15-17). The methodcontinues in one or more branches. Along one branch, the initialcoordinates of the player are determined using one or more of aplurality of position determining techniques as described herein. Thisbranch continues by updating the player's position to track the player'smotion using one or more of a plurality of motion tracking techniques asdescribed herein.

The other branch includes determining the coordinates of the gamingobject's initial position using one or more of a plurality of positiondetermining techniques as described herein. This branch continues byupdating the gaming object's position to track the gaming object'smotion using one or more of a plurality of motion tracking techniques asdescribed herein. Note that the rate of tracking the motion of theplayer and/or gaming object may be done at a rate based on the videogaming being played and the expected speed of motion. Further note thata tracking rate of 10 milliseconds provides 0.1 mm accuracy in motiontracking.

FIG. 22 is a diagram of another method for determining position and/ormotion tracking that begins by determining a reference point within acoordinate system (e.g., the vector coordinate system of FIGS. 18-20).The reference point may be the origin or any other point within thelocalized physical area. The method continues in one or more branches.Along one branch, a vector with respect to the reference point isdetermined to indicate the player's initial position, which may be doneby using one or more of a plurality of position determining techniquesas described herein. This branch continues by updating the player'sposition to track the player's motion using one or more of a pluralityof motion tracking techniques as described herein.

The other branch includes determining a vector with respect to thereference point for the gaming object to establish its initial position,which may be done by using one or more of a plurality of positiondetermining techniques as described herein. This branch continues byupdating the gaming object's position to track the gaming object'smotion using one or more of a plurality of motion tracking techniques asdescribed herein. Note that the rate of tracking the motion of theplayer and/or gaming object may be done at a rate based on the videogaming being played and the expected speed of motion. Further note thata tracking rate of 10 milliseconds provides 0.1 mm accuracy in motiontracking.

FIG. 23, FIG. 24, and FIG. 25 are diagrams of another embodiment of acoordinate system of a localized physical area that may be used for agaming system. In these diagrams, an xyz origin is selected to besomewhere in the localized physical area and the initial position of apoint being tracked on the player and/or gaming object is determinedbased on its Cartesian coordinates (e.g., x1, y1, z1). As the playerand/or gaming object moves, the new position of the tracking and/orpositioning points are determined in Cartesian coordinates with respectto the preceding location (e.g., Δx, Δy, Δz).

As another example, the positioning and motion tracking of the playermay be done with reference to the position of the gaming object, suchthe gaming objects position is determined with reference to the originand/or its previous position and the position of the player is determinewith reference to the gaming object's position. The reverse could beused as well. Further, both position and motion of the gaming object andthe player may be referenced to a personal item of the player, such as acell phone.

FIG. 26, FIG. 27, and FIG. 28 are diagrams of another embodiment of acoordinate system of a localized physical area that may be used for agaming system. In these diagrams, an origin is selected to be somewherein the localized physical area and the initial position of a point beingtracked on the player and/or gaming object is determined based on itsvector, or spherical coordinates (e.g., ρ1, φ1, θ1). As the playerand/or gaming object moves, the new position of the tracking and/orpositioning points are determined as a vector, or spherical coordinateswith respect to the preceding location (e.g., ΔV, or Δρ, Δφ, Δθ).

As another example, the positioning and motion tracking of the playermay be done with reference to the position of the gaming object, suchthe gaming objects position is determined with reference to the originand/or its previous position and the position of the player is determinewith reference to the gaming object's position. The reverse could beused as well. Further, both position and motion of the gaming object andthe player may be referenced to a personal item of the player, such as acell phone.

FIG. 29 is a diagram of another method for determining position and/ormotion tracking that begins by determining environment parameters of thephysical area in which the gaming object lays and/or in which the gamesystem lays. The environmental parameters include, but are not limitedto, height, width, and depth of the localized physical area, objects inthe physical area, differing materials in the physical area, multiplepath effects, interferers, etc.

The method then proceeds by mapping the environment parameters to acoordinate system (e.g., one of the systems shown in FIGS. 15-17). As anexample, if the physical area is a room, a point in the room is selectedas the origin and the coordinate system is applied to at least some ofthe room. In addition, objects in the room (e.g., a couch, a chair,etc.) are mapped to the coordinate system based on their physicallocation in the room.

The method then proceeds by determining the coordinates of the player's,or players', position in the physical area. The method then continues bydetermining the coordinates of a gaming object's initial position. Notethat the positioning of the gaming object may be used to determine theposition of the player(s) if the gaming object is something worn by theplayer or is close proximity to the player. Alternatively, the initialposition of the player may be used to determine the initial position ofthe gaming object. Note that one or more of the plurality of positioningtechniques described herein may be used to determine the position of theplayer and/or of the gaming object.

The method then proceeds by updating the coordinates of the player's, orplayers', position in the physical area to track the player's motion.The method also continues by updating the coordinates of a gamingobject's position to track its motion. Note that the motion of thegaming object may be used to determine the motion of the player(s) ifthe gaming object is something worn by the player or is close proximityto the player. Alternatively, the motion of the player may be used todetermine the motion of the gaming object. Note that one or more of theplurality of motion techniques described herein may be used to determinethe position of the player and/or of the gaming object.

FIG. 30 is a diagram of another method for determining position and/ormotion tracking that begins by determining a reference point within thephysical area in which the gaming object lays and/or in which the gamesystem lays. The method then proceeds by determining a vector for aplayer's initial position with respect to a reference point of acoordinate system (e.g., one of the systems shown in FIGS. 18-20). As anexample, if the physical area is a room, a point in the room is selectedas the origin and the coordinate system is applied to at least some ofthe room.

The method then continues by determining a vector of a gaming object'sinitial position. Note that the positioning of the gaming object may beused to determine the position of the player(s) if the gaming object issomething worn by the player or is close proximity to the player.Alternatively, the initial position of the player may be used todetermine the initial position of the gaming object. Note that one ormore of the plurality of positioning techniques described herein may beused to determine the position of the player and/or of the gamingobject.

The method then proceeds by updating the vector of the player's, orplayers', position in the physical area to track the player's motion.The method also continues by updating the vector of the gaming object'sposition to track its motion. Note that the motion of the gaming objectmay be used to determine the motion of the player(s) if the gamingobject is something worn by the player or is close proximity to theplayer. Alternatively, the motion of the player may be used to determinethe motion of the gaming object. Note that one or more of the pluralityof motion techniques described herein may be used to determine theposition of the player and/or of the gaming object.

FIG. 31 is a diagram of another method for determining position and/ormotion tracking that begins by determining environment parameters of thephysical area in which the gaming object lays and/or in which the gamesystem lays. The environmental parameters include, but are not limitedto, height, width, and depth of the localized physical area, objects inthe physical area, differing materials in the physical area, multiplepath effects, interferers, etc.

The method then proceeds by mapping the environment parameters to acoordinate system (e.g., one of the systems shown in FIGS. 23-25). As anexample, if the physical area is a room, a point in the room is selectedas the origin and the coordinate system is applied to at least some ofthe room. In addition, objects in the room (e.g., a couch, a chair,etc.) are mapped to the coordinate system based on their physicallocation in the room.

The method then proceeds by determining the coordinates of the gamingobject's initial position in the physical area. The method thencontinues by determining the coordinates of the player's initialposition with respect to the gaming object's initial position. Note thatone or more of the plurality of positioning techniques described hereinmay be used to determine the position of the player and/or of the gamingobject.

The method then proceeds by updating the coordinates of the gamingobject's position in the physical area to track its motion. The methodalso continues by updating the coordinates of the player's position totrack the player's motion with respect to the gaming object. Note thatone or more of the plurality of motion techniques described herein maybe used to determine the position of the player and/or of the gamingobject.

FIG. 32 is a diagram of another method for determining position and/ormotion tracking that begins by determining a reference point within thephysical area in which the gaming object lays and/or in which the gamesystem lays. The method then proceeds by determining a vector for agaming object's initial position with respect to a reference point of acoordinate system (e.g., one of the systems shown in FIGS. 26-28). As anexample, if the physical area is a room, a point in the room is selectedas the origin and the coordinate system is applied to at least some ofthe room.

The method then continues by determining a vector of the player'sinitial position with respect to the gaming object's initial position.Note that one or more of the plurality of positioning techniquesdescribed herein may be used to determine the position of the playerand/or of the gaming object.

The method then proceeds by updating the vector of the gaming object'sposition in the physical area to track its motion. The method alsocontinues by updating the vector of the player's position with respectto the gaming object's motion to track the player's motion. Note thatone or more of the plurality of motion techniques described herein maybe used to determine the position of the player and/or of the gamingobject.

FIG. 33 is a diagram of another embodiment of a coordinate system of agaming system that is an extension of the coordinate systems discussedabove. In this embodiment, the coordinate system includes a positioningcoordinate grid and a motion tracking grid, where the motion trackinggrid is of a finer resolution than the positioning coordinate grid. Ingeneral, the player or gaming object's position within the physical areacan have a first tolerance (e.g., within a meter) and the motiontracking of the player and/or the gaming object has a second tolerance(e.g., within a few millimeters). As such, the position of the playerand/or gaming object can be updated infrequently in comparison to theupdating of the motion (e.g., the position can be updated once everysecond or so while the motion may be updated once every 10milliseconds).

FIG. 34 is a diagram of a method for determining motion of a gamingobject and/or a player that begins by determining an initial position ofthe player and/or gaming object using one or more of the positioningtechniques described herein. The method continues by determining motionreference points for the player and/or for the gaming object as shown inFIG. 35. The reference points may be sensors on the player and/or on thegaming object, may be particular body parts (e.g., nose, elbow, knee,etc.), particular points on the gaming object, and/or a combinationthereof. The number of reference points and the location thereof may bedependent on the video game, on the player's physical characteristics,on the player's skill level, on the desired motion tracking resolution,and/or on the motion tracking technique being used.

The method continues by determining initial motion coordinates for eachreference point using one or more the position determining techniquesand/or motion tracking techniques described herein. The method continuesby establishing one or more data rates for the reference points based onthe location of the reference point, motion patterns (e.g., a videobowling game, the player will have particular motions for bowling),previous motion (e.g., half way through bowling a ball, know where thenext motion is likely to be), and/or human bio-mechanics (e.g., arms andlegs bends in a certain manner). For example, the reference point of ahand may have a faster data rate than a reference point on the headsince the hand will most likely being moving faster and more often thanthe head.

The method continues by obtaining motion tracking data (e.g., distances,vectors, distance changes, vector changes, etc.) for the referencepoints at intervals of the one or more data rates. The method continuesby determining motion of the reference points based on the motiontracking date at intervals of the one or more data rates.

FIG. 36, FIG. 37, FIG. 38, and FIG. 39 are diagrams of examples ofmotion patterns in accordance with human bio-mechanics. As shown in FIG.36, a head can move up/down, it can tilt, it can rotate, and/or acombination thereof. For a given video game, head motion can beanticipated based on current play of the game. For example, during anapproach shot, the head will be relatively steady with respect totilting and rotating, and may move up or down along with the body.

FIG. 37 shows the motion patterns of an arm (or leg) in accordance withhuman bio-mechanics. As shown, the arm (or leg) may contract or extend,go up or down, move side to side, rotate, or a combination thereof. Fora given video game, an arm (or leg) motion can be anticipated based onthe current play of the game. Note that the arm (or leg) may be brokendown in smaller body parts (e.g., upper arm, elbow, forearm, wrist,hand, fingers). Further note that the gaming object's motion will besimilar to the body part it is associated with.

FIG. 38 illustrates the likely motions of a torso, which can moveup/down, side to side, front to back, and/or a combination thereof. Fora given video game, torso motion can be anticipated based on currentplay of the game. As such, based on the human bio-mechanical limitationsand ranges of motion along with the video game being player, the motionof the player and/or the associated gaming object may be anticipated,which facilitates better motion tracking.

FIG. 39 is a diagram of an example of motion estimation for the head,right arm, left arm, torso, right leg, and left leg of a video gameplayer. In this game, it is anticipated that the arms will move the mostoften and over the most distance, followed by the legs, torso, and head.In this example the interval rate may be 10 milliseconds, which providesa 1 mm resolution for an object moving at 200 miles per hour. In thisexample, the body parts are not anticipated to move at or near 200 mph.

At interval 1, at least some of the reference points on thecorresponding body parts is sampled. Note that each body part mayinclude one or more reference points. Since the arms are anticipated tomove the most and/or over the greatest distances, the reference point(s)associated with the arms are sampled once every third interval (e.g.,interval 1, 4, 7). For intervals 2 and 3, the motion of the referencepoints is estimated based on the samples of intervals 1 and 4 (and maybe more samples at different intervals), the motion pattern of the arm,human bio-mechanics, and/or a combination thereof. The estimation may bea linear estimation, a most likely estimation, and/or any othermathematical technique for estimating data points between two or moresamples. A similar estimation is made for intervals 5 and 6.

The legs have a data rate of sampling once every four intervals (e.g.,intervals 1, 5, 9, etc.). The motion data for the intervening intervalsis estimated in a similar manner as the motion data of the arms wasestimated. The torso has a data rate of sampling once every five samples(e.g., interval 1, 6, 11, etc.). The head has a data rate of samplingonce every six samples (e.g., interval 1, 7, 13, etc.). Note that theinitial sampling does not need to be done during the same interval forall of the reference points.

FIG. 40 and FIG. 41 are diagrams of examples of reference points on aplayer to determine player's physical measurements. In this example,once the positioning of the reference points is determined, theirpositioning may be used to determine the physical attributes of theplayer (e.g., height, width, arm length, leg length, shoe size, etc.).

FIG. 42 is a diagram of an example of mapping a player to an image ofthe video game. In this embodiment, the image displayed in the videogame corresponds to the player such that, as the player moves, the imagemoves the same way. The image may a stored image of the actual player, acelebrity player (e.g., a professional athlete), a default image, and/ora user created image. The mapping involves estimating motion of thenon-reference points of the player based on the reference points of theplayer. In addition, the mapping involves equating the reference pointson the player to the same points on the image. The same may be done forthe gaming object.

FIG. 43 is a diagram of another method for determining motion thatbegins by obtaining coordinates for the reference points of the playerand/or gaming object. The method continues by determining the player'sdimensions and/or determining the dimensions of the gaming object. Themethod continues by mapping the reference points of the player tocorresponding points of a video image based on the player's dimensions.This step may also include mapping the reference points of the gamingobject (e.g., a sword) to the corresponding image of the gaming objectbased on the gaming object's dimensions.

The method continues by determining coordinates of other non-referencedbody parts and/or parts of the gaming object based on the coordinates ofthe reference points. This may be done by a linear interpolation, by amost likely motion algorithm, by a look up table, and/or any othermethod for estimated data points from surrounding data points. Themethod continues by tracking motion of the reference points andpredicting motion of the non-referenced body parts and/or parts of thegaming object based on the motion of the reference points. This may alsobe done by a linear interpolation, by a most likely motion algorithm, bya look up table, and/or any other method for estimated data points fromsurrounding data points.

FIG. 44 is a schematic block diagram of an embodiment of a gaming objectand/or game console that includes a physical layer (PHY) integratedcircuit (IC) and a medium access control (MAC) layer processing module.The PHY IC includes a position and/or motion tracking RF section, acontroller interface RF section, and a baseband processing module. Aslike any processing module disclosed herein, the MAC processing moduleand the baseband processing module may be a single processing device ora plurality of processing devices. Such a processing device may be amicroprocessor, micro-controller, digital signal processor,microcomputer, central processing unit, field programmable gate array,programmable logic device, state machine, logic circuitry, analogcircuitry, digital circuitry, and/or any device that manipulates signals(analog and/or digital) based on hard coding of the circuitry and/oroperational instructions. The processing module may have an associatedmemory and/or memory element, which may be a single memory device, aplurality of memory devices, and/or embedded circuitry of the processingmodule. Such a memory device may be a read-only memory, random accessmemory, volatile memory, non-volatile memory, static memory, dynamicmemory, flash memory, cache memory, and/or any device that storesdigital information. Note that when the processing module implements oneor more of its functions via a state machine, analog circuitry, digitalcircuitry, and/or logic circuitry, the memory and/or memory elementstoring the corresponding operational instructions may be embeddedwithin, or external to, the circuitry comprising the state machine,analog circuitry, digital circuitry, and/or logic circuitry. Furthernote that, the memory element stores, and the processing moduleexecutes, hard coded and/or operational instructions corresponding to atleast some of the steps and/or functions illustrated in the variousFigures depicted and described herein.

The MAC processing module triggers position and/or tracking datacollection, formatting of the data, processing of the data, and/orcontrolling position and/or tracking data communications and/orcontroller communications. The position and/or tracking RF section mayinclude circuitry to transmit one or more beamformed RF signals, RFsignals for 3D antenna reception, RFID communications, and/or any otherRF transmission and/or reception discussed herein.

The game console may use a standardized protocol, a proprietaryprotocol, and/or a combination thereof to provide the communicationbetween the gaming object and the console. Note that the communicationprotocol may borrow unused bandwidth from a standardized protocol tofacilitate the gaming communication (e.g., utilize unused BW of a WLAN,cell phone, etc.).

FIG. 45, FIG. 46, and FIG. 47 are diagrams of various embodiments ofmethods for determining position and/or motion tracking.

Referring to the method of FIG. 45, the method operates by capturingdigital images using multiple digital cameras. The method then performsprocessing of the digital images to identify characteristics of anobject that is depicted within at least some of the digital images. Themethod then operates by determining position of the object based on theidentified characteristics. This determined position is with respect tothe locations of at least some of the multiple digital cameras.

Once the position of the object is known, the method can continue bymapping this determined position to a virtual 3D (three-dimensional)coordinate system.

Referring to the method of FIG. 46, the method operates by capturingdigital images using multiple digital cameras. The method then performsprocessing of the digital images to identify characteristics of anobject that is depicted within at least some of the digital images. Oncethese characteristics of the object are identified, the method operatesby generating directional vectors based on the identifiedcharacteristics. These directional vectors may be viewed as extendingfrom locations of at least some of the multiple digital cameras to aposition of the object.

The method then operates by determining position of the object based onthe directional vectors. Again, this determined position is with respectto the locations of at least some of the multiple digital cameras asindicated by an intersection of at least some of the directionalvectors.

Once the position of the object is known, the method can continue bymapping this determined position to a 3D (three-dimensional) coordinatesystem.

Referring to the method of FIG. 47, the method operates by capturingdigital images using multiple digital cameras. The method then performsprocessing of the digital images to identify at least one sensing tagthat is depicted within at least some of the digital images. The sensingtag can be any of a variety of sensing tags, including a lightreflective material, a light absorbent material, am infrared source(e.g., when at least one of the digital cameras is infrared sensitive),a color, and/or any other desired type of sensing tag. The sensing tagmay be associated with an entirety of object depicted within at leastsome of the digital images. As also described herein, the sensing tagmay be associated with only a portion of an object associated depictedwithin at least some of the digital images (e.g., a corner of an object,a body part of a player, etc.).

Once the sensing tag is identified within at least some of the digitalimages, the method operates by generating directional vectors based onthe identified sensing tag. These directional vectors may be viewed asextending from locations of at least some of the multiple digitalcameras to a position of the sensing tag.

The method then operates by determining position of the sensing tagbased on the directional vectors. Again, this determined position iswith respect to the locations of at least some of the multiple digitalcameras as indicated by an intersection of at least some of thedirectional vectors.

Once the position of the object is known, the method can continue bymapping this determined position to a virtual 3D (three-dimensional)coordinate system.

FIG. 48 is a diagram of an embodiment of a method for determining adistance based on captured digital images.

Referring to the method of FIG. 48, the method operates by capturingdigital images using multiple digital cameras. The method then performsprocessing of the digital images, using pattern recognition, to identifyan object depicted within at least some of the digital images. A size ofthe identified object is predetermined (e.g., such as a predeterminedsize of a gaming object, a known object, etc.).

In accordance with processing the digital images, the method operates todetermine an image size of the identified object (e.g., a size of theobject as depicted within at least one of the digital images). Once animage size of an object depicted within a digital image is know, andalso when an actual size of the object is known, then the method canassociate the known/predetermined size with the image size. This way, ascaling factor can be determined between objects depicted within thedigital image and the actual size of objects within the a physicalenvironment that includes the object.

The method then operates by determining a distance within the physicalenvironment using the image size of the object and the predeterminedsize of the object (e.g., based on the scaling factor).

Again, this determined position is with respect to the locations of atleast some of the multiple digital cameras as indicated by anintersection of at least some of the directional vectors. Once adistance, as depicted within at least one digital image is known, thenthe method can continue by mapping this determined distance within avirtual 3D (three-dimensional) coordinate system.

It is noted that the various modules (e.g., processing modules, basebandprocessing modules, MAC processing modules, game consoles, etc.)described herein may be a single processing device or a plurality ofprocessing devices. Such a processing device may be a microprocessor,micro-controller, digital signal processor, microcomputer, centralprocessing unit, field programmable gate array, programmable logicdevice, state machine, logic circuitry, analog circuitry, digitalcircuitry, and/or any device that manipulates signals (analog and/ordigital) based on operational instructions. The operational instructionsmay be stored in a memory. The memory may be a single memory device or aplurality of memory devices. Such a memory device may be a read-onlymemory, random access memory, volatile memory, non-volatile memory,static memory, dynamic memory, flash memory, and/or any device thatstores digital information. It is also noted that when the processingmodule implements one or more of its functions via a state machine,analog circuitry, digital circuitry, and/or logic circuitry, the memorystoring the corresponding operational instructions is embedded with thecircuitry comprising the state machine, analog circuitry, digitalcircuitry, and/or logic circuitry. In such an embodiment, a memorystores, and a processing module coupled thereto executes, operationalinstructions corresponding to at least some of the steps and/orfunctions illustrated and/or described herein.

The present invention has also been described above with the aid ofmethod steps illustrating the performance of specified functions andrelationships thereof. The boundaries and sequence of these functionalbuilding blocks and method steps have been arbitrarily defined hereinfor convenience of description. Alternate boundaries and sequences canbe defined so long as the specified functions and relationships areappropriately performed. Any such alternate boundaries or sequences arethus within the scope and spirit of the claimed invention.

The present invention has been described above with the aid offunctional building blocks illustrating the performance of certainsignificant functions. The boundaries of these functional buildingblocks have been arbitrarily defined for convenience of description.Alternate boundaries could be defined as long as the certain significantfunctions are appropriately performed. Similarly, flow diagram blocksmay also have been arbitrarily defined herein to illustrate certainsignificant functionality. To the extent used, the flow diagram blockboundaries and sequence could have been defined otherwise and stillperform the certain significant functionality. Such alternatedefinitions of both functional building blocks and flow diagram blocksand sequences are thus within the scope and spirit of the claimedinvention.

One of average skill in the art will also recognize that the functionalbuilding blocks, and other illustrative blocks, modules and componentsherein, can be implemented as illustrated or by discrete components,application specific integrated circuits, processors executingappropriate software and the like or any combination thereof.

Moreover, although described in detail for purposes of clarity andunderstanding by way of the aforementioned embodiments, the presentinvention is not limited to such embodiments. It will be obvious to oneof average skill in the art that various changes and modifications maybe practiced within the spirit and scope of the invention, as limitedonly by the scope of the appended claims.

1. An apparatus, comprising: a plurality of digital cameras thatgenerates a plurality of digital images, wherein an object is depictedwithin at least some of the plurality of digital images; and aprocessing module coupled to: receive the plurality of digital images;identify characteristics of the object within the at least some of theplurality of digital images to produce identified objectcharacteristics; and determine position of the object with respect tothe plurality of digital cameras based on the identified objectcharacteristics.
 2. The apparatus of claim 1, wherein: the identifiedobject characteristics includes a plurality of directional vectorsextending from at least some of the plurality of digital cameras to theobject; and the processing module determines the position of the objectwith respect to the plurality of digital cameras based on the pluralityof directional vectors.
 3. The apparatus of claim 2, wherein: one of theplurality of directional vectors extends from a reference point of adigital image sensor of one digital camera to a physical pixel withinthe digital image sensor that corresponds to an image pixel of onedigital image captured by the one digital camera.
 4. The apparatus ofclaim 1, wherein: the processing module employs a pattern recognitionprocess to identify the characteristics of the object.
 5. The apparatusof claim 1, wherein: the object includes a sensing tag; and at least oneof the identified object characteristics is the sensing tag.
 6. Theapparatus of claim 5, wherein: the sensing tag is at least one of: alight reflective material; a light absorbent material; an infraredsource such that at least one of the plurality of digital cameras isinfrared sensitive; and a color.
 7. The apparatus of claim 1, wherein:the object has a predetermined size; the processing module employs apattern recognition process to identify the object within the at leastsome of the plurality of digital images; the identified object has animage size; and based on the predetermined size and the image size, theprocessing module determines a distance between the object and theprocessing module or at least one of the plurality of digital cameras.8. The apparatus of claim 1, wherein: the processing module maps theposition of the object within a virtual three-dimensional coordinatesystem.
 9. The apparatus of claim 1, wherein: a field of view of onecamera of the plurality of digital cameras is adjusted based on theposition of the object.
 10. The apparatus of claim 1, wherein: an imagecapture rate of one of the plurality of digital cameras is adjustedbased on at least one of: a predetermined setting within the processingmodule; a user-selected setting within the processing module; a movementhistory of the object; a current movement of the object; and an expectedfuture movement of the object.
 11. The apparatus of claim 1, wherein:the processing module determines the position of the object during afirst time; the processing module determines at least one additionalposition of the object during a second time; and the processing moduleestimates movement of the object by comparing the determined positionand the at least one additional determined position.
 12. The apparatusof claim 1, wherein: the object includes a first radio frequency (RF)transceiver; the processing module includes a second RF transceiver; andbased on an RF signal transmitted from the first RF transceiver to thesecond RF transceiver, the processing module determines a distancebetween the processing module and the object.
 13. The apparatus of claim1, wherein: one of the plurality of digital cameras includes a firstradio frequency (RF) transceiver; the processing module includes asecond RF transceiver; and based on an RF signal transmitted from thefirst RF transceiver to the second RF transceiver, the processing moduledetermines a distance between the processing module and the one digitalcamera.
 14. The apparatus of claim 1, wherein: a plurality of integratedcircuits is distributed throughout a region in which the object islocated; and one of the plurality of digital cameras is a digital imagesensor implemented on a surface of one of the plurality of integratedcircuits.
 15. An apparatus, comprising: a gaming object for use within agaming environment; a plurality of digital cameras that generates aplurality of digital images, wherein the gaming object is depictedwithin at least some of the plurality of digital images; and a gameconsole coupled to: receive the plurality of digital images; identifycharacteristics of the gaming object within the at least some of theplurality of digital images to produce identified objectcharacteristics; and determine position of the gaming object within thegaming environment with respect to the plurality of digital camerasbased on the identified object characteristics.
 16. The apparatus ofclaim 15, wherein: the gaming object is associated with a player locatedwithin the gaming environment; and the game console determines positionof the player based on the position of the gaming object.
 17. Theapparatus of claim 15, wherein: the identified object characteristicsincludes a plurality of directional vectors extending from at least someof the plurality of digital cameras to the gaming object; and the gameconsole determines the position of the gaming object with respect to theplurality of digital cameras based on the plurality of directionalvectors.
 18. The apparatus of claim 17, wherein: one of the plurality ofdirectional vectors extends from a reference point of a digital imagesensor of one digital camera to a physical pixel within the digitalimage sensor that corresponds to an image pixel of one digital imagecaptured by the one digital camera.
 19. The apparatus of claim 15,wherein: the game console employs a pattern recognition process toidentify the characteristics of the gaming object.
 20. The apparatus ofclaim 15, wherein: the gaming object includes a sensing tag; and atleast one of the identified object characteristics is the sensing tag.21. The apparatus of claim 20, wherein: the sensing tag is at least oneof: a light reflective material; a light absorbent material; an infraredsource such that at least one of the plurality of digital cameras isinfrared sensitive; and a color.
 22. The apparatus of claim 15, wherein:the gaming object has a predetermined size; the game console employs apattern recognition process to identify the gaming object within the atleast some of the plurality of digital images; the identified gamingobject has an image size; and based on the predetermined size and theimage size, the game console determines a distance between the gamingobject and the game console or at least one of the plurality of digitalcameras.
 23. The apparatus of claim 15, wherein: the game console mapsthe position of the gaming object within a virtual three-dimensionalcoordinate system.
 24. The apparatus of claim 15, wherein: an imagecapture rate of one of the plurality of digital cameras is adjustedbased on at least one of: a predetermined setting within the gameconsole; a player-selected setting within the game console; a movementhistory of the gaming object; a current movement of the gaming object;and an expected future movement of the gaming object.
 25. The apparatusof claim 15, wherein: the position is a first position; the game consoledetermines the first position during a first time; the game consoledetermines a second position of the gaming object during a second time;and the game console estimates movement of the gaming object bycomparing the first position and the second position.
 26. An apparatus,comprising: a plurality of digital cameras, associated with a gamingobject, that generates a plurality of digital images such that aplurality of predetermined references is depicted within at least someof the plurality of digital images; and a game console coupled to:receive the plurality of digital images; identify characteristics of atleast some of the plurality of predetermined references to produceidentified characteristics; and determine position of the gaming objectwith respect to the plurality of predetermined references based on theidentified characteristics.
 27. The apparatus of claim 26, wherein: theidentified characteristics includes a plurality of directional vectorsextending from at least some of the plurality of digital cameras to theat least some of the plurality of predetermined references; and the gameconsole determines the position of the gaming object with respect to theplurality of digital cameras based on the plurality of directionalvectors.
 28. The apparatus of claim 27, wherein: one of the plurality ofdirectional vectors extends from a physical pixel within a digital imagesensor of one digital camera, that corresponds to an image pixel of onedigital image captured by the one digital camera, to a reference pointof the digital image sensor.
 29. The apparatus of claim 26, wherein: thegaming object is associated with a player located within the gamingenvironment; and the game console determines position of the playerbased on the position of the gaming object.
 30. The apparatus of claim26, wherein: the game console employs a pattern recognition process toidentify the characteristics of at least some of the plurality ofpredetermined references.
 31. The apparatus of claim 26, wherein: thegame console maps the position of the gaming object within a virtualthree-dimensional coordinate system.
 32. The apparatus of claim 26,wherein: the position is a first position; the game console determinesthe first position during a first time; the game console determines asecond position of the gaming object during a second time; and the gameconsole estimates movement of the gaming object by comparing the firstposition and the second position.